Regional Educational 
Laboratory Northeast 


Institute of APPENDICES 
=o [ULer=) dco) ameter (=) alersss5 June 2021 


At Education Development Center 


Accelerated college credit programs and their relationship 
imom-relelor-halolar-lm-lae-linlan-lel mem avarele(-m Ci r-lale| 


Appendix A. About the study 
Appendix B. Methods 

Appendix C. Supporting analysis 
Appendix D. Other analyses 


See https://ies.ed.gov/ncee/edlabs/projects/project.asp?projectID=5680 for the full report. 


Appendix A. About the study 


Many states have adopted accelerated college credit programs that allow high school students to simultaneously 
earn credit toward a high school diploma and toward a postsecondary degree as a strategy to improve students’ 
college readiness and college success. All but three states have a statewide policy promoting accelerated college 
credit programs (Zinth, 2016a). Further, 27 states, including Rhode Island, made dual enrollment or college credits 
part of the “fifth indicator” in their Every Student Succeeds Act plans (Jordan & Marley, 2018). Whereas some 
programs allow high school students to take regular college classes taught by faculty on a college campus and 
earn credits toward both their high school and their college degrees (referred to as dua! enrollment in this study), 
others offer college credit-bearing courses taught by a high school instructor on-site at a high school (referred to 
here as concurrent enrollment). Other high school-based programs, such as Advanced Placement (AP) and 
International Baccalaureate courses, allow students to earn college credits through a standardized curriculum by 
scoring well on an assessment that is recognized by many colleges in lieu of credits. Some schools offer college 
credit as part of a comprehensive program, such as the early college model, through which high schools and 
postsecondary institutions partner to provide historically underrepresented students with an integrated course 
sequence and comprehensive academic and socioemotional support to make substantial progress toward a 
college credential during high school (Zinth, 2016b). International Baccalaureate and early college programs are 
not widely offered in Rhode Island—no public schools offer International Baccalaureate courses, and annually 
there are approximately 100 students enrolled in early college programs based on the P-TECH model that aims for 
students to complete a two-year college degree during high school. 


Educational outcomes of accelerated college credit programs 

Rigorous quasi-experimental design studies have found positive impacts of accelerated college credit programs 
on college enrollment, persistence, and degree completion (An, 2013; Giani et al., 2014; Struhl & Vargas, 2012).° 
Experimental design studies of the early college model have linked participation to increased rates of high school 
graduation, college enrollment, and college completion (Edmunds et al., 2017; Haxton et al., 2016). A randomized 


5 These three quasi-experimental studies each used a propensity score—based approach similar to the one employed in the current study, 
with some differences in specification. For example, An (2013) used propensity score weights rather than matching; Struhl and Vargas 
(2012) selected one-to-one matched comparison students only from districts offering the accelerated college credit treatment; and Giani 
et al. (2014) drew one-to-one matched comparison students from districts not offering accelerated college credit options. 
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controlled trial of a concurrent enrollment mathematics course had mixed effects, demonstrating a positive impact 
on subsequent rigorous mathematics course-taking during high school and on enrolling in a four-year rather than 
two-year college but showing no significant effect on college enrollment in general or on college mathematics 
performance (Hemelt, Schwartz, & Dynarski, 2019). Evidence from a correlational study also suggests a positive 
association between engaging in accelerated college credit programs and avoiding development education course 
enrollment in college (Davis, Smither, Zhu, & Stephan, 2017). 


Few large-scale studies have assessed the comparative value of different accelerated college credit models. Many 
studies on the topic do not distinguish between courses taught at high schools and at colleges, or between courses 
taught by high school or by college instructors. The location of course offerings is important because high school 
and college-based programs offer distinct advantages and disadvantages, such as different required qualifications 
for instructors, opportunities for students to develop familiarity with the college setting, and transportation and 
geographic barriers (Piontek, Kannapel, Flory, & Stewart, 2016; Zinth & Barnett, 2018). Two quasi-experimental 
design analyses comparing college credit-bearing courses of unspecified type to AP courses found divergent 
results, with An (2013) identifying no difference in college degree attainment in a nationally representative sample 
but Giani et al. (2014) estimating a larger effect on postsecondary outcomes associated with college credit-bearing 
courses than with AP course-taking among Texas students. However, few studies—and no rigorous impact 
studies—were identified that compared the effects of dual and concurrent enrollment models. One statewide 
correlational study found that concurrent enrollment courses had larger positive effects on postsecondary 
enrollment than AP courses, whereas dual enrollment courses had no effect (Speroni, 2011). A second 
correlational study examining dual, concurrent, and AP participation suggested a possible positive association 
between each of the programs and faster bachelor’s degree completion (Burns, Ellegood, Bernard Bracy, Duncan, 
& Sweeney, 2019). 


Without careful thought about eligibility requirements and program characteristics, the benefits of accelerated 
college credit programs may apply only to academically advanced students who would have attended college 
anyway (Zinth & Barnett, 2018). In many states, accelerated college credit programs serve a student population 
that is disproportionately White, female, academically high achieving, and not economically disadvantaged (Davis 
et al., 2017; Estacion, Cotner, D’Souza, Smith, & Borman, 2011; Lochmiller, Sugimoto, Muller, Mosier, & 
Williamson, 2016; Pierson, Hodara, & Luke, 2017). If students are not adequately prepared for challenging college- 
level courses, accelerated college credit programs may even have negative effects on students’ educational 
outcomes (Miller et al., 2018). 


Using a quasi-experimental design and a statewide sample, this study contributes rigorous evidence concerning 
the effects of accelerated college credit courses on students’ educational attainment in high school and college. 
In addition, the study adds exploratory findings to the limited evidence base on differences in educational 
outcomes associated with dual course enrollment, concurrent course enrollment, and AP test-taking. It also 
examines the relationship of these programs to college readiness as measured by developmental education 
course enrollment. 
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Appendix B. Methods 


This appendix describes the data and analysis methods used in the study. 


Data 


Data for this study drew from administrative data sources for a statewide longitudinal sample of Rhode Island 
public high school students. 


Sample 


The study focuses on a cohort of students who were first-time ninth graders in 2013/14. The Rhode Island 
Department of Education (RIDE) began collecting detailed, reliable student-level data on dual and concurrent 
enrollments in 2015/16. The 2013/14 grade 9 cohort was the first for which the state agency had two years of 
these data. The study focused on this cohort because at the time, it was the only cohort for which the state had 
complete data on dual and concurrent enrollment for both the 2015/16 and 2016/17 school years (their on-time 
junior and senior years) as well as data on postsecondary outcomes from the state longitudinal data system 
through fall 2019. Two additional cohorts (the grade 9 cohorts of 2014/15 and 2015/16) were analyzed as a 
supplemental component of the study, as described in appendix D. 


The sample included all public school students who met the cohort definition, with the exceptions described here. 
Students were included if they attended a public charter school, specialty school, or vocational-technical school; 
if they dropped out or left the state’s public schools after grade 9; and if they were not continuously enrolled in 
the years after grade 9. Students were excluded from the sample if they were enrolled in grade 9 in 2013/14 but 
were repeating the grade the next year; if their grade 9 enrollment record was less than 30 days long; or if their 
grade 9 enrollment was not associated with a specific high school, as is sometimes the case with students receiving 
outplacements for special services. Because the study used a quasi-experimental approach that matched students 
on their characteristics before they started high school, the sample was also limited to students who had an 
enrollment record for grade 8 in the previous year (2012/13) and a valid standardized state assessment score 
(values greater than or equal to 800) for that year. These exclusions reduced the sample from 9,915 students 
enrolled in grade 9 for the first time in 2013/14 to 8,726 students in 62 schools. The excluded students were more 
likely to be racial/ethnic minority, economically disadvantaged, and multilingual learners, in part reflecting the 
differences in characteristics of students from immigrant families who enter the Rhode Island public schools after 
grade 8 or whose families move more frequently. 


Variables 


All measures were provided by DataHUB, the state’s longitudinal data system, which houses and links data 
elements from a wide range of state agencies and other sources. For this study, the sources (as indicated in table 
B1) were RIDE, the Rhode Island Office of the Postsecondary Commissioner (RIOPC), and the National Student 
Clearinghouse (NSC). 
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Table B1. Measures used in the study 


Measure How used in study NYolU] get-o) me l-}-) 
Enrollment in dual enrollment course in grades 11-12? Treatment RIDE 
Enrollment in concurrent enrollment course in grades 11-12? Treatment RIDE 
Took an Advanced Placement test in grades 9-12 Treatment RIDE 

High school diploma received Outcome: High school RIDE 

completion 

Enrollment in any postsecondary institution within 12 months Outcome: College enrollment NSC and RIOPC 
of high school graduation date 

Developmental education course enrollment record (any Outcome: College readiness RIOPC 


subject) at a Rhode Island public college within one year of high 
school graduation 
Covariate for: 


propensity treatment 
score effect 

estimation estimation 
Student 
Eligible for National School Lunch Program (an indicator of Yes Yes RIDE 
economic disadvantage) (moderator) 
Scaled math score for New England Common Assessment Yes Yes RIDE 
Program, grade 8 
Racial/ethnic minority (yes/no) Yes Yes RIDE 
Sex (male, female) Yes RIDE 
Multilingual learner, grade 8 Yes RIDE 
Has an Individualized Education Program, grade 8 Yes RIDE 
Chronically absent (>= 10 percent of days), grade 8 Yes RIDE 
Suspended one or more times, grade 8 Yes RIDE 
School 
School of student’s first grade 9 enrollment record (Level-2 Yes Yes 
cluster membership indicator) 
Total enrollment Yes RIDE 
Driving distance to nearest Rhode Island public college campus Yes RIDE 
Locale (urban, suburban, or rural) Yes RIDE 
Percent students eligible for National School Lunch Program Yes RIDE 
Percent students who are multilingual learners Yes RIDE 
Percent students who have an individualized education Yes RIDE 
program 


NSC is National Student Clearinghouse. RIDE is Rhode Island Department of Education. RIOPC is Rhode Island Office of the Postsecondary Commissioner. 
School-level measures reflect 2013/14 values. 

Note: ? Dual and concurrent enrollment data were available for grades 11-12 for the 2013/14 grade 9 cohort featured in the body of the report, grades 10— 
12 for the 2014/15 grade 9 cohort, and grades 9-12 for the 2015/16 grade 9 cohort (see appendix D for the latter cohort analyses). 


Treatment status—participation in an accelerated college credit program—was defined as having one or more 
enrollment records for dual or concurrent enrollment courses in grades 11-12, and a record of taking one or more 
AP tests in grades 9-12. This approach has the following limitations: 


e Because reliable student-level data on dual and concurrent enrollment were available starting in 2015/16, 
participation in accelerated college credit programs was captured only for grades 11 and 12. Based ona 
later cohort (2015/16 grade 9 cohort) that has these data spanning the full four years of high school, an 
estimated 2 percent of students take dual and concurrent enrollment courses in grades 9 or 10 but not in 
later grades, meaning that they would be incorrectly classified as not participating in such courses in this 
study. 
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Dual enrollment records are available only from the state’s public colleges. Rhode Island also funds dual 
enrollment at private colleges in the state but does not collect student-level enrollment information from 
these institutions. Aggregate figures provided by RIDE indicate that approximately 1,000 high school 
students per year take courses at the three participating private colleges in the state. If these students did 
not enroll in additional courses at Rhode Island public colleges, they would be misidentified as 
nonparticipants in this study, potentially attenuating the estimated effects of the program. 


RIDE does not collect data on AP course enrollment—only on tests taken. If students enrolled in an AP 
course but did not take the test, they would be misidentified as not participating in this type of accelerated 
college credit program. 


RIDE does not identify the approximately 100 students per year who enroll in early college programs in 
the state. Although these students’ participation in dual or concurrent enrollment is captured at the 
course level, the effects of this more comprehensive model for accelerated college credit cannot be 
differentiated. 


College outcome measures were drawn from a combination of national and state sources. 


NSC collects enrollment and degree completion data on approximately 98 percent of the nation’s students 
at public and private postsecondary institutions, including less-than-two-year institutions (National 
Student Clearinghouse, 2020). If an enrollment record was not found in NSC or in the Rhode Island public 
college enrollment dataset, the research team coded the student as not college enrolled. However, 
students may be missing from NSC records incorrectly because the student or institution opts out of data 
reporting or because of errors in matching records. 


RIOPC collects data from the state’s three public colleges: the Community College of Rhode Island and 
two four-year institutions, the University of Rhode Island, and Rhode Island College. In some cases, a 
student had an enrollment record reported by RIOPC but no record from NSC. These students were 
counted as enrolled in college in the year of the RIOPC record. If a student’s only enrollment records were 
dated before the student’s high school graduation date (suggesting dual or concurrent enrollment), the 
student was coded as not college enrolled. 


Developmental education course enrollment data are available from RIOPC but not from NSC. Inferences 
drawn from the analyses of developmental education course enrollment are valid only for students who 
enrolled in a Rhode Island public college. At the time of the analysis, such enrollments were available only 
for the student’s first year after graduating from high school, meaning that students who placed into 
developmental education but delayed enrolling would be misclassified as college ready. 


Missing data 


After applying other exclusions discussed above, 3 percent of students were missing a valid New England Common 


Assessment Program (NECAP) score for grade 8. No other data were missing. Given the low level of missingness, 
students with missing NECAP values were excluded from analysis. 


Descriptive analysis of program participation rates 


Participation rates were calculated for participation in any accelerated college credit program as well as for each 


of the three programs individually, and were disaggregated by student and school characteristics. These 


descriptive analyses were performed on the unmatched sample. Results indicated that 41 percent of students 


participated in one or more accelerated college credit programs, with differences in participation rates by program 
and by student characteristics (tables C1 through C6 and C9). 
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Propensity score matching analysis 

This study used observational data; students in the study sample chose whether to participate in the accelerated 
college credit programs and were not randomly assigned to participate. Because students self-selected into the 
programs, observed differences in outcomes for those who chose to enroll and those who did not could be due 
to initial differences between these groups that led them to select into or out of the programs, rather than to the 
programs themselves. To reduce this potential selection bias, the research team used a propensity score matching 
approach to construct a comparison group of students who were statistically similar to the treatment students 
before starting high school (Rosenbaum & Rubin, 1985). 


First, researchers estimated the probability that each student would participate in accelerated college credit 
programs. This propensity score was derived from a two-level logistic regression model with a random effect for 
school. The models estimated the student’s observed treatment status as a function of a set of variables for 
student characteristics measured before the student entered high school and characteristics of the first high 
school the student attended (see table B1). A two-level model was chosen in part because school factors 
constrained the student’s decision to participate in the programs. In particular, eight schools did not offer 
concurrent enrollment. The predictors selected were correlated with treatment status and with the outcome 
measures of interest (Austin, 2011). Once the propensity models were estimated, the propensity score was 
calculated as a predicted probability of treatment, using both fixed effects in the model as well as the school 
random effect from the empirical Bayes means. Although the distributions of the resulting propensity scores for 
treatment and comparison students overlapped between the values of 0.01 and 0.95, which is the area of common 
support, the ratio of treatment to comparison students was very high at the upper end of the distribution with 
propensity scores above 0.60, and very low at the opposite end of the distribution with propensity scores below 
0.20. Among treatment students, 52 had propensity scores higher than the area of common support (see figure 
B1). 


Figure B1. Distribution of propensity scores for treatment and comparison students; 2013/14 grade 9 cohort 
Frequency of propensity scores 
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Note: The sample size is 8,726 students who were first-time ninth graders in Rhode Island public schools in 2013/14. See table C2 in appendix C for more 
information. 
Source: Authors’ analysis of data from Rhode Island Department of Education. 


Students who participated in accelerated college credit programs were then matched with students who did not 
based on the closeness of their propensity scores. The resulting sample was a group of students who enrolled in 
an accelerated college credit program—the treatment group—and a group of students with similar predicted 
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probabilities of enrolling, but who did not enroll—the comparison group. Comparisons of outcomes for these two 
groups provide an estimate of the average treatment effect on the treated (ATT). ATT represents the difference 
between what actually happened to students in the accelerated programs when they participated and what would 
have happened had they not participated. 


The research team tried several different matching specifications (Guo & Fraser, 2010), including many-to-one 
matching, whereby multiple comparison students are matched to a treatment student; matching with 
replacement, in which comparison students can be matched to more than one treatment student; and preferential 
matching, which first looks for an acceptable match within the same school and then searches at other schools if 
necessary (Arpino & Cannas, 2016). The specification resulting in the best baseline equivalence was a nearest 
neighbor algorithm specifying one-to-one matching without replacement and limiting matches within a caliper of 
0.20 standard deviations of the propensity score, which allowed students to be matched with peers attending 
different schools (psmatch2 in Stata version 14). The matching process resulted in a reduced sample of 1,958 
treatment and 1,958 comparison students, a total sample of 3,916 students in 57 schools. Among the treatment 
students, 1,628 (45 percent) were excluded because there were not enough acceptable matches among the 
comparison students to pair with them. Researchers chose this conservative matching approach because it 
achieved greater internal validity through the greatest reduction in selection bias; however, this decision reduced 
external validity, since the higher-scoring treatment students were excluded and inferences about those students 
could not be made (Austin, 2011). It should be noted that in matching some treatment students with a comparison 
student outside their school, the analysis did not entirely hold school effects constant between the treatment and 
comparison groups. 


After matching, the resulting treatment and comparison groups were found to be statistically similar on baseline 
characteristics, with standardized mean differences of less than 0.05 (Rubin, 2001; What Works Clearinghouse, 
2020; see table B2). 
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Table B2. Baseline equivalence before and after matching, 2013/14 grade 9 cohort 


Before matching After matching 
Comparison Standardized Treatment Comparison Standardized 
Treatment mean (n = mean (n= mean mean(n= mean(n= mean 
3,586) oye 0)) difference 1,958) 1,958) (olhixelactalers) 

Student level: 
NECAP (grand 5.66 -4.91 1.02 1.25 1.64 -0.04 
mean centered) 
Has IEP 0.04 0.23 -0.57 0.07 0.07 0.00 
Multilingual 0.03 0.06 -0.16 0.05 0.04 0.02 
learner 
National School 0.28 0.50 -0.48 0.38 0.36 0.03 
Lunch Program 
eligible 
Chronic absence 0.07 0.23 -0.44 0.12 0.11 0.03 
(10+) 
Has 1 or more 0.03 0.15 -0.43 0.03 0.05 0.01 
suspensions 
Male 0.42 0.58 -0.33 0.49 0.47 0.04 
Racial/ethnic 0.29 0.39 -0.19 0.34 0.34 0.00 
minority 
School level: 
Number of 6.81 6.81 0.00 6.79 6.80 -0.03 
students enrolled 
(log value) 
Driving distance 6.36 5.84 0.52 6.18 6.18 0.00 
to nearest Rhode 
Island public 
college campus 
Percent students 0.36 0.45 -0.09 0.41 0.40 0.01 
eligible for 


National School 
Lunch Program 


Percent students 0.03 0.04 -0.01 0.03 0.03 0.00 
who are 

multilingual 

learners 

Percent students 0.13 0.15 -0.02 0.14 0.14 0.00 
who have an IEP 

Locale (suburban) 0.68 0.66 0.02 0.64 0.66 -0.02 
Locale (rural) 0.10 0.09 0.01 0.12 0.11 0.01 


IEP is individualized education program. NECAP is New England Common Assessment Program. 
Source: Authors’ analysis of data from Rhode Island Department of Education. 


The three accelerated college credit programs were combined for the purpose of identifying matches. Although 
55 percent of treatment students overall were matched with a comparison student, the proportion differed across 
the three programs. Although 55 percent of accelerated college credit program participants were matched overall, 
the matching rates differed across the three individual programs. Among dual enrollment students, 66 percent 
were matched; 55 percent of concurrent enrollment students were matched; and 47 percent of AP test-takers 
were matched. 


The baseline characteristics of students in each program before and after matching are displayed in table B3. 
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Table B3. Baseline characteristics of treatment groups by program, 2013/14 grade 9 cohort 
WW/Ke¥e Tals oX=) Ke) com aakelmel allay 


Student level: 
NECAP (grand 
mean 
centered) 
Has IEP 
Multilingual 
learner 
National 
School Lunch 
Program 
eligible 
Chronic 
absence (10+) 
Has 1 or more 
suspensions 
Male 
Racial/ethnic 
minority 
School level: 
Number of 
students 
enrolled (log 
value) 
Driving 
distance to 
nearest RI 
public college 
Percent 
students 
eligible for 
National 
School Lunch 
Program 
Percent 
students 
Multilingual 
learners 
Percent 
students with 
IEP 

Locale 
(suburban) 
Locale (rural) 


Ever dual 
yale) | {exe} 
(n = 336) 


0.51 
0.08 


0.05 


0.56 


0.17 


0.09 
0.32 


0.53 


6.55 


4.09 


0.56 


0.03 


0.14 


0.38 
0.06 


Ever 
concurrent 
Tale) | (=1e} 
(n = 2,223) 


5.93 
0.04 


0.03 


0.23 


0.07 


0.03 
0.43 


0.24 


6.82 


6.70 


0.33 


0.02 


0.14 


0.75 
0.12 


Ever took 

Advanced 

Placement 
test 

(n = 2,367) 


7.89 
0.02 


0.02 


0.24 


0.05 


0.02 
0.41 


0.27 


6.84 


6.69 


0.35 


0.03 


0.13 


0.69 
0.10 


Comparison 
(n = 5,140) 


-4.91 
0.23 


0.06 


0.50 


0.23 


0.15 
0.58 


0.39 


6.81 


5.84 


0.45 


0.04 


0.15 


0.66 
0.09 


>AV=l ae [UT] 
Tale) | (=1e} 
(n = 221) 


-2.86 
0.12 


0.07 


0.62 


0.24 


0.13 
0.32 


0.56 


6.58 


4.26 


0.56 


0.04 


0.14 


0.41 
0.05 


IEP is individualized education program. NECAP is New England Common Assessment Program. 
Note: Students who participated in any accelerated college credit program were matched with students who did not participate in any of the three programs. 
Some students participated in more than one program. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 


Means after matching 


Ever 
concurrent 
Tal ae) | (Xe 
(n = 1,228 


1.58 
0.07 


0.05 


0.33 


0.12 


0.05 
0.50 


0.28 


6.82 


0.37 


0.03 


0.15 


0.70 
0.14 


lAV{=l gm Kolo) 4 

No N-larecye| 

Placement 
test 

(n = 1,104) 


3.36 
0.04 


0.04 


0.35 


0.09 


0.04 
0.51 


0.33 


6.81 


6.53 


0.41 


0.04 


0.14 


0.61 
0.13 


Comparison 
mean 
(n = 1,958) 


1.64 
0.07 


0.04 


0.36 


0.11 


0.05 
0.47 


0.34 


6.80 


6.18 


0.40 


0.03 


0.14 


0.66 
0.11 


The first research question was addressed using a series of two-level logistic regression models in the matched 
sample, where the outcome was a binary indicator of attaining the given outcome and treatment was represented 
by a binary indicator for ever having participated in any of the three accelerated college credit programs. 


The main analysis model was: 
Student-level model: 
Log(P / (1— P))i = bo; + bi(Any Accelerated Program), + b«(Student-level covariates); 
School-level model: 
Model for intercept 
boj = Goo + Gai(School-level covariates); + Uo 
Models for slopes 
by = 910 
by = Gao 


where P is the probability of a student’s attaining a given educational outcome, subscripts j and j refer to individual 
student j in high school j, c is a vector of student-level covariates, u is the school random effect, and d is a vector 
of school-level covariates. Slopes were fixed. 


The decision to use hierarchical generalized linear modeling as opposed to single-level regression with clustered 
standard errors was based on the ability to obtain more precise estimates, and the flexibility to evaluate the 
sensitivity of estimates to different model parameters and assumptions. The intraclass correlation coefficient 
indicated that 9 percent of variation in high school graduation was attributable to between-school differences.® 
District was not included as a third level in the model because districts did not explain substantially more outcome 
variation than high schools alone. 


The model produces results in terms of odds ratios. In this study, the odds ratios represent the change in the odds 
of a student’s attaining a given outcome, such as graduation from high school, in relation to whether the student 
participated in an accelerated college credit program. A statistically significant odds ratio above 1 indicates that 
participation is related to increased odds of a student’s attaining the outcome compared with the odds for a 
student who did not participate, and a statistically significant odds ratio below 1 indicates decreased odds of 
attainment for the participating student. 


This report also translates estimated odds ratios into predicted probabilities that use the estimated regression 
coefficients to produce a point estimate of the probability of attaining the outcome conditional on treatment 
status. Specifically, the study team calculated two predicted probabilities for each student: one under the 
assumption that the student was treated and the other under the assumption that the student was untreated. 
We then averaged the predicted probabilities under each scenario across all students (Pampel, 2020). 


Three baseline student and school measures were included as covariates because they were tested for moderating 
effects: eligibility for National School Lunch Program, the measure used to represent economic disadvantage; 
grade 8 standardized test score (New England Common Assessment Program [NECAP] mathematics, grand-mean 
centered); and racial/ethnic minority. In addition, three school-level covariates were included: percentage of 
students eligible for National School Lunch Program, percentage of multilingual learners, and percentage of 
students with individualized education programs (IEPs). These variables were included because when the model 


6 The intraclass correlation coefficient was estimated using the Snijders and Bosker (2012) formula for binary outcomes too/(too + 12/3) 
where too is the estimated between-school variance component in the unconditional two-level model with random intercept for school. 
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was applied to a second cohort (2014/15 grade 9 cohort) as an additional exploratory analysis, the standardized 
mean differences after matching were greater than 0.05 for these baseline characteristics as follows: grade 8 
NECAP mathematics score (-0.06), percentage of students in the school eligible for national school lunch program 
(0.06), percentage of students in the school who were multilingual learners (0.08), and percentage of students in 
the school who had an IEP (0.08). What Works Clearinghouse guidelines (2020) require the inclusion of covariates 
to reduce the remaining selection bias from such factors. 


The regression analyses indicated positive treatment effects of accelerated college credit program participation 
on high school graduation and college enrollment within one year of high school (tables C7 and C8). 


As asensitivity analysis, different sets of covariates were included in the models: none; NECAP mathematics score 
only; propensity score only; and all covariates that had been used in propensity score estimation. Additional 
sensitivity analyses included a fixed effect for schools rather than a random effect, and regression in the 
unmatched sample. The direction, magnitude, and significance of the treatment coefficient remained very similar 
in all specifications of the model. 


An additional sensitivity analysis was performed to assess whether the findings might be attributable to early 
drop-out. If students left the Rhode Island public school system before grade 11, they did not have the opportunity 
to enroll in dual or concurrent enrollment courses that were measured for this study. (Such students could take 
dual or concurrent enrollment in earlier grades, but data were not available for those years; students could also 
be counted as treatment participants if they took Advanced Placement (AP) tests in earlier grades.) Researchers 
excluded the 350 students with no enrollment record in either 2015/16 or 2016/17 (the years of on-time 
enrollment in grades 11 and 12 for the study cohort), re-estimated propensity scores, performed matching based 
on the new scores, and then estimated treatment effects using the new matched sample. The direction and the 
statistical significance of the treatment coefficients remained similar to those estimated from the original sample, 
although their magnitude was somewhat smaller (table C16). 


Moderation effects 


To examine the moderating effect of student economic disadvantage, the main model analyzing program effects 
was expanded to include its interaction with the treatment indicator. The analysis model is shown below. 


Student-level model: 
Log(P /(1—P)) = 


boj + ba(Any Accelerated Program);+ b.(Student-level covariates); + b2(Econ Dis) + b3(Any Accelerated Program )j*(Econ 
Dis) ij 


School-level model: 
Model for intercept 

boj = Joo + Gar(School-level covariates); + Uo 
Model for slopes bi; to b3 

Bmj = mo 


In the model above, the moderating effect of economic disadvantage is reflected in b3, the coefficient for the 
treatment-by-economic-disadvantage interaction. The analyses did not find any evidence for such a moderating 
effect for the outcomes examined (tables C12 and C13). 


The research team also examined the moderating effect of race/ethnicity, defined as “White” or “racial/ethnic 
minority.” The results revealed that racial/ethnic minority status moderated the treatment effect in two out of 
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the three outcomes considered: College enrollment and enrollment in developmental education courses. 
Specifically, the benefit of accelerated college credit program participation was larger for racial/ethnic minority 
students than for White students for college enrollment. Treatment significantly increased the predicted 
probability of not taking developmental education courses at a Rhode Island public college for both White and 
racial/ethnic minority students; it decreased the predicted probability of taking developmental education courses 
at a Rhode Island public college for White students, but did not alter this probability for racial/ethnic minority 
students. Meanwhile, treatment significantly increased the predicted probability of enrolling in a college other 
than a Rhode Island public college for both White and racial/ethnic minority students and significantly decreased 
the predicted probability of not enrolling in any college for both White and racial/ethnic minority students. 
Racial/ethnic minority status did not moderate the treatment effect on high school graduation (see tables C17 
and C18). 


Comparison among types of accelerated college credit programs 
To estimate the relationship between each specific accelerated college credit program and nonparticipating 
comparison students, regression analyses were performed on the same matched sample as was used for the 
overall accelerated college credit program treatment estimation model. 


Researchers considered using a more tailored matching approach, for example by matching dual enrollment 
students with concurrent enrollment students. However, such an approach was deemed impractical because 
some students participated in more than one of the three programs, and because the student and high school 
characteristics of dual enrollment students differed substantially from those of concurrent and AP students. 


To compare the relative size of effects among the three types of accelerated programs, the main model analyzing 
program effects was expanded as follows. The treatment indicator in the model was replaced by a series of three 
indicators: ever enrolled in a dual enrollment course, ever enrolled in a concurrent enrollment course, and ever 
took an AP test. The model is shown below. 


Student-level model: 
Log(P / (1— P))j = bo; + bi(Ever Dual); + b2(Ever Concurrent); + b3(Ever AP); + b-(Student-level covariates); 
School-level model: 
Model for intercept 
bo = Goo + gai(School-level covariates); + Uo 
Model for slopes bito b,j 
Bj = Gmo 


In the model above, the effect of each type of accelerated program is reflected in the coefficient bm that is 
associated with the program type. Pairwise post-estimation tests were performed to assess whether the 
estimated coefficients for each of the three programs were significantly different from each other. Results 
indicated positive relationships between each program and each outcome (table C15). The post-estimation tests 
identified a larger effect for Advanced Placement than for dual enrollment on college enrollment. 


Several points are noteworthy for this analysis. First, in this cohort in Rhode Island, dual enrollment occurred 
almost exclusively at the state’s two-year public college, whereas concurrent enrollment occurred primarily at the 
state’s four-year public colleges. Thus, the effects of the college are confounded with the effects of the type of 
accelerated college credit program. 
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Second, the analysis was performed using a matched sample, which enabled the estimation of the effects of these 
three types of accelerated programs as a whole. However, whether this analysis is capable of estimating the effect 
of each program type separately is debatable, because the matched comparison group represents those who did 
not take any of the accelerated programs rather than those who did not take that particular program type. 


Third, the three types of accelerated programs were coded in such a manner that they are not mutually exclusive. 
For example, a student who takes more than one type of accelerated programs may have “yes” for “ever 
participated in dual enrollment” and for “ever took an AP test.” A consequence of this coding scheme is that the 
estimated effect of each program type is calculated using course-taking behavior rather than the student as a unit. 
An alternative coding scheme would be to divide the treatment students into mutually exclusive and exhaustive 
groups consisting of “only participated in dual enrollment,” “only participated in concurrent enrollment, 
took an AP test,” and “participated in multiple programs.” This alternative coding would estimate the effect of 
each program type using student as a unit. 


vu 


only 


A sensitivity analysis was performed using this alternative mutually exclusive coding scheme. These alternative 
analyses produced results similar to those of the original model in terms of the direction, magnitude, and 
significance of the estimated effect of each of the three types of accelerated program, with the following 
exception: participating in more than one program was associated with larger positive effects on each of the 
outcomes than was participating in any one program alone. 


Treatment effect estimation for developmental education course enrollment outcome 


The effect of treatment on developmental education course enrollment was assessed using a multinomial 
regression model. At first glance, developmental course enrollment might appear to be a binary outcome. A 
student either enrolled in a developmental education course or did not enroll in a developmental education 
course. If this had been the case, the model analyzing program effects used for other binary outcomes would have 
sufficed for this analysis as well. 


However, as mentioned in the data section, information on developmental education course enrollment was 
available only for students who enrolled in Rhode Island public colleges. For this reason, the outcome was coded 
to have four potential values: enrolling in a Rhode Island public college and taking developmental education 
courses, enrolling in a Rhode Island public college and not taking developmental education courses, enrolling in 
another type of college, and not enrolling in college. This coding scheme allowed for the use of all students in the 
propensity score matched analytic sample, preserving the quasi-experimental design capable of estimating 
treatment effects. Had the analysis been performed using only the subsample of students who enrolled in Rhode 
Island public colleges, the subsets of treatment and matched comparison students could no longer be assumed to 
represent statistically equivalent groups. Furthermore, because the treatment affected the probability of enrolling 
in Rhode Island public colleges in the first place, such an analysis would have produced biased results. 


Like logistic models, multinomial models are fit to predict log-odds of the probability of target event. However, 
unlike logistic models, multinomial models must handle multiple target events. Consequently, in multinomial 
models, one target event is chosen as the base category. Then a set of equations is fit to predict the log-odds of 
the probability of each remaining target event over the probability of the base category. The model analyzing 
program effects used for developmental education course enrollment is shown below. The right-hand side of the 
model consists of the same set of predictors as in the multilevel logistic model used for the effect estimate for 
other outcomes, with one exception. Because the statistical software used for this analysis, Stata version 14, did 
not have a multilevel option for multinomial models, a single-level multinomial model was used with cluster- 
adjusted standard errors, which are also robust to heteroskedasticity (White, 1980). 
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Student-level model: 
1) Log(P_DevEdRI/P_NoDevEdRI);= bo_veveari + D1_peveari(Any Accelerated Program); + be _peveari(Student-level covariates); 


2) Log(P_NoCollege / P_NoDevEdRI); = bo Nocoltege + D1_Nocollege(Any Accelerated Program); + be nocoltege(Student-level 
covariates); 


3) Log(P_OtherCollege / P_NoDevEdRI); = bo_othercollege + D1_ othercollege(Any Accelerated Program); + be othercollege(Student- 
level covariates); 


where for example P_DevEdRI / P_NoDevEdRI is the probability of a student’s taking developmental education 
courses at a Rhode Island public college over the probability of the student’s not taking development education 
courses at a Rhode Island public college. 


The nonlinear form of the coefficients in multinomial models, called relative risk ratios (RRR), were converted to 
predicted probabilities for this report just as the odds ratios were for the logistic models. 


One note of caution in interpreting predicted probabilities in the results of multinomial analysis: when the 
outcome consists of mutually exclusive and exhaustive categories, interpreting the predicted probability for one 
outcome category in isolation could be misleading. For example, this study found that treatment is associated 
with a relatively small difference in the predicted probability of developmental education course enrollment (14 
percent compared with 17 percent) but also found that treatment is associated with large increases in the 
predicted probability of enrolling in college in the first place (tables C10 and C11). In short, predicted probabilities 
from multinomial analyses must be interpreted in the context of the predicted probabilities of all outcome 
categories. 
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Appendix C. Supporting analysis 
This appendix contains tables displaying the results of analyses on the 2013/14 grade 9 cohort in more detail. 


Table C1. Accelerated college credit program participation, 2013/14 grade 9 cohort 
All (@(o) | [=-X-Molg=tolim o)ge)-4e-]eaM ey-)aa(el]ey-1aleln 


students Any college credit Any concurrent Took any Advanced 
in cohort program Any dual enrollment large) iiaatsiaue Placement test 


N N Percent N Percent N Percent N Percent 


2013/14 


8,726 3,586 41.1 336 3.9 2,223 25.5 2,367 27.1 
grade 9 cohort 


AP is Advanced Placement. 
Note: Dual and concurrent enrollment data are only available from 2015/16 and 2016/17, whereas AP test data are available from all four years. 
Source: Authors’ analysis of data for 2013/14 through 2016/17 from Rhode Island Department of Education. 


Table C2. Accelerated college credit program student characteristics, 2013/14 grade 9 cohort 
(Go) | (-F-X-Melg=toli mo) go)-4e-1n 0M oy-) aor) yyy a) 


Took any 
Characteristics in All students in Na Ware) I(=-Xomelg eye i Any dual Any concurrent PXeNFTater=te| 
Ag-le(oms) program enrollment enrollment Placement test 
Percent Percent Percent 

Male 4,454 51.0 1,490 41.6 108 32.1 948 42.7 975 41.2 
acta ethnic 3,039 348 1,057 29.5 177 52.7 530 23.8 648 27.4 
minority 
National School 
Lunch Program 4,144 47.5 1,193 33.3 211 62.8 631 28.4 676 28.6 
eligible 
Has IEP 1,327 15.2 149 4.2 27 8.0 91 4.1 53 22 
Mupeilingtl 423 4.6 105 2.9 18 5.4 65 2.9 54 2.3 
learner 
pee or mele 908 10.4 115 3.2 30 8.9 60 2.7 50 24 
suspensions 
Chronic absence 

1,448 16.6 268 75 56 16.7 162 7.3 124 5.2 
(10+) 
NECAP math: 
eupsrentally 2,045 23.4 289 8.1 69 20.5 167 75 106 4.5 
below Proficient 
Parley 1,618 186 393 11.0 65 19.4 225 10.1 180 76 
Proficient 
Proficient 3,695 42.3 1,786 49.8 146 43.5 1,119 50.3 1,124 47.5 
Ererielene with 1,361 15.6 1,117 31.2 55 16.4 712 32.0 957 40.4 


Distinction 


AP is Advanced Placement. IEP is individualized education program. N is number of students. NECAP is New England Common Assessment Program. 
Note: The sample size is 8,726 students who were first-time ninth graders in 2013/14. The sample size is 8,719 for NECAP math because there are seven 
cases with NECAP scores but no proficiency level. 


Source: Authors’ analysis of data from Rhode Island Department of Education. 
Es 


Table C3. Accelerated college credit program students by the characteristics of their first high school, 2013/14 
grade 9 cohort 


(Go) |(=¥-x-Molg=tol im o)ae)-4e-]aam ox-]ad iol] ey-14(0) a) 


All students in Na Waere)|(-14-melg=tel ia Any dual Any concurrent 

Nol atoXe) el a¥-ler-(eik Tai (03 (oto) ale)a program enrollment enrollment Took any AP test 

N Percent Percent Percent 
Locale 
Rural 828 9.5 366 10.2 20 6.0 273 12.3 229 9.7 
Suburban 5,808 66.6 2,438 68.0 127 37.8 1,661 74.7 1,629 68.8 
Urban 1,916 22.0 751 20.9 183 54.5 269 12.1 485 20.5 
Urban ring 174 2.0 31 0.9 6 1.8 20 0.9 24 1.0 
Driving distance to nearest Rhode Island public college campus 
<=1mile 
ae 537 6.2 161 4.5 98 29.2 35 1.6 47 2.0 
distance) 
1-8 miles 5,581 64.0 2,279 63.6 183 54.5 1,449 65.2 1,494 63.1 
>8 miles 2,608 29.9 1,146 32.0 55 16.4 739 33.2 826 34.9 
Enrollment 
Small 707 8.1 290 8.1 55 16.4 190 8.6 134 5.7 
Medium 2,461 28.2 2,303 64.2 236 70.2 1,297 58.3 1,646 69.6 
Large 5,558 63.7 993 27.7 45 13.4 736 33.1 587 24.8 
Star school accountability rating (5 indicates best rating) 
1 1,351 15.9 396 11.2 61 18.9 169 7.7 265 11.4 
2 1,729 20.4 532 15.1 124 38.5 346 15.7 262 11.3 
3 2,693 31.8 972 27.6 69 21.4 663 30.1 630 27.2 
4 2,255 26.6 1,273 36.1 61 18.9 899 40.8 852 36.7 
5 450 5.3 352 10.0 7 2.2 125 5.7 310 13.4 


AP is Advanced Placement. 

Note: The sample size is 8,726 students who were first-time ninth graders in 2013/14. The sample size is 8,478 for Star rating because 248 students attended 
schools with no Star rating. N is number of students. State agencies in Rhode Island commonly use the term “urban” to refer to the cities of Central Falls, 
Newport, Pawtucket, Providence, and Woonsocket, and “urban ring” to refer to the surrounding municipalities of Cranston, East Providence, North 
Providence, Warwick, and West Warwick. Enrollment categories are defined thus: small equals less than 500 students, medium equals 500-1,199, and large 
equals 1,200 or higher. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 
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Table C4. Economically disadvantaged students: Accelerated college credit program student characteristics, 


2013/14 grade 9 cohort 


College credit program participation 


Xoreyareyany cer) 1 
disadvantaged 


students in Na Waero) | (124) Took any 
cohort credit Any dual Any concurrent NeW Tayercro| 
Characteristics in grade 8 (N = 4,144) program enrollment enrollment Placement test 
Percent 

Male 2,126 51.3 449 37.6 66 31.3 230 36.5 264 39.1 
Racial Grins 2,467 59.5 798 66.9 158 74.9 380 60.2 A66 68.9 
minority 
Has IEP 859 20.5 72 6.0 18 8.5 41 6.5 20 3.0 
Mulblingual 383 9.2 88 7.4 14 6.6 55 8.7 48 7A 
learner 
Heer es 706 17.0 87 73 25 11.9 42 6.7 35 5.2 
suspensions 
ne ppeetce 1,063 25.7 174 14.6 46 21.8 92 14.6 75 da 
NECAP math: 
ones ely 1,518 36.7 219 18.4 55 26.1 120 19.0 89 13.2 
below Proficient 
Partially Proficient 936 22.6 218 18.27 52 24.6 101 16.0 110 16.3 
Proficient 1,452 35.1 598 50.1 90 42.7 311 49.3 353 52.2 
pheneleat with 233 516 158 13.2 14 6.6 99 15.7 124 18.3 


Distinction 


AP is Advanced Placement. IEP is individualized education program. N is number of students. NECAP is New England Common Assessment Program. 
Note: The sample size is 4,144 students who were first-time ninth graders in 2013/14 and eligible for the National School Lunch Program. The sample size is 


4,139 for NECAP math. There are five cases with NECAP scores but no proficiency level. 
Source: Authors’ analysis of data from Rhode Island Department of Education. 


Table C5. Economically disadvantaged students: Accelerated college credit program students by the 
characteristics of their first high school, 2013/14 grade 9 cohort 


(Go) | K-¥-x-Molg=tol im olqoy-4e-)aam oy-Vad iol] oy-ha (ola) 
Economically 
disadvantaged 


Kyelstere)| students in cohort Pa Waere)|(-1-4-melg tel ia Any dual Any concurrent 

characteristics (N = 4,144) program enrollment enrollment Took any AP test 
N Percent N Percent Percent Percent Percent 

Locale 

Rural 207 5.0 59 5.0 4 2.0 44 7.0 29 4.3 

Suburban 2,443 59.0 585 49.0 56 26.5 413 65.5 308 45.6 

Urban 1,418 34.2 541 45.4 149 70.6 169 26.8 336 49.7 

Urban ring 76 1.8 8 1.0 2 1.0 5 1.0 3 0.0 

Driving distance to nearest Rhode Island public college campus 

<=1mile 

a 449 10.8 138 11.6 81 38.4 32 5.1 43 6.4 

distance) 

1-8 miles 2,878 69.5 883 74.0 115 54.5 483 76.6 536 79.3 

>8 miles 817 19.7 172 14.4 15 7.1 116 18.4 97 14.4 

Enrollment 

Small 442 10.7 158 13.2 42 20.0 80 12.7 69 10.2 

Medium 2,712 65.4 818 68.6 149 70.6 379 60.1 522 77.2 

Large 990 23.9 217 18.2 20 9.5 172 27.3 85 12.6 

Star school accountability rating (5 indicates best rating) 

1 1,195 29.9 341 29.5 52 25.9 141 22.5 225 34.6 

2 998 25.0 254 22.0 93 46.3 139 22.2 104 16.0 

3 1,191 29.8 293 25.3 39 19.4 213 34.0 130 20.0 

4 460 11.5 146 12.6 11 5.5 101 16.1 79 12.1 

5 155 3.9 123 10.6 6 3.0 32 5.1 113 17.4 


AP is Advanced Placement. N is number of students. 

Note: The sample size is 4,144 students who were first-time ninth graders in 2013/14 and eligible for the National School Lunch Program. The sample size is 
3,999 for Star rating because 145 students attended schools with no Star rating. State agencies in Rhode Island commonly use the term “urban” to refer to 
the cities of Central Falls, Newport, Pawtucket, Providence, and Woonsocket, and “urban ring” to refer to the surrounding municipalities of Cranston, East 
Providence, North Providence, Warwick, and West Warwick. Enrollment categories are defined thus: small equals less than 500 students, medium equals 
500-1,199, and large equals 1,200 or higher. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 
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Table C6. Observed percentage of students attaining outcomes, 2013/14 grade 9 cohort 
(@(o) | (-¥-x-Morg=tol im o]go)-4e-] 00M ey-] ad lol] eyo) a) 


Took any 

All students in Any college credit Any dual Any concurrent Noh FTavercte| 
Outcomes cohort fo) ge)-4e-)00) enrollment enrollment Placement test 

N Percent Percent 
Total 8,726 100.0 3,586 41.1 336 3.9 2,223 25.5 2,367 27.1 
Graduated 
from high 7,150 81.9 3,458 96.4 322 95.8 2,140 96.3 2,287 96.6 
school 
Enrolled in 
college 
within one 5,366 61.5 3,155 88.0 270 80.4 1,999 90.0 2,165 91.5 
year of 
graduation 


AP is Advanced Placement. N is number of students from a cohort who were first-time ninth graders in 2013/14. 
Source: Authors’ analysis of data from Rhode Island Department of Education. 


Table C7. Regression coefficients (odds ratios), 2013/14 grade 9 cohort 


High school graduation Enrollment in college 
Silalerlage! Syelalerlae! 

Variable Odds ratio error (ofole Ka g-hale) error 
Student-level 
Intercept 8.67*** 3.17 1.75** 0.32 
Participation in any accelerated college credit 8.69*** 1.20 5.21*** 0.42 
program 
NECAP math score 0.99 0.01 1.03*** 0.01 
Eligible for National School Lunch Program 0.68** 0.09 0.62*** 0.06 
Racial/ethnic minority 0.73* 0.11 1.16 0.13 
School-level 
Percent students eligible for National School Lunch 0.70 0.40 0.66 0.20 
Program 
Percent multilingual learner students 0.01* 0.03 0.19 0.20 
Percent students with IEP 0.12 0.25 0.19 0.53 
Variance components 
Between-school variance (Too) 0.49 0.13 0.06 0.03 


* p< .05; ** p< .01; *** p< .001. 

IEP is individualized education program. NECAP is New England Common Assessment Program. 

Note: The sample size is 3,916 students who were first-time ninth graders in 2013/14 clustered in 57 high schools. Shown are estimated coefficients from 
two-level logistic regression models, fit separately for each of the two outcomes. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 


Table C8. Predicted probabilities from regression of any accelerated college credit program participation on 
each outcome, 2013/14 grade 9 cohort 


CTg-1o[UF-yu Yo alt -4amcyel aloye) | Enrolled in college within one year of graduation 
WEeTarl eo) (=) Probability S\iclalerlacm-lanele Probability Sliclalel-laemlanele 
Participation in any accelerated college credit program 
Yes 0.96*** 0.01 0.84*** 0.01 
No 0.75 0.02 0.54 0.01 


* p< .05; ** p< .01; *** p< .001. 


Note: The sample size is 3,916 students clustered in 57 high schools in a matched sample drawn from students who were first-time ninth graders in Rhode 
Island public schools in 2013/14. Predicted probabilities are estimated from the regression analysis displayed in table C7. The model included student and 
school covariates (not shown in this table) as listed in table B1, appendix B. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 


Table C9. Observed rate of enrollment during first year of college in developmental education courses by 


accelerated college credit program participation, among students who enrolled at a Rhode Island public 
college, 2013/14 grade 9 cohort 


(Go) | (-¥-X-Morg=tol im o)ge)-4e-] a0 ey-] adel] oy-s (ola) 
All students in 


(oto) ato) am -talae)i (=e Mla} Took any 
RI public colleges Any college credit Any dual Any concurrent Ne WeTarvecto| 
(n = 3392) program Yalae)iiant=iais yalae)iiaat=lale Placement test 
Outcomes (n = 1707) (n = 198) (n = 1062) Cee Keley) 
Percent 

Enrolled in 
development 4969: a7 331 19.4 62 31.3 189 17.8 140 213.1 
al education 
course(s) 


AP is Advanced Placement. N is number of students within a cohort of first-time ninth graders in 2013/14. Developmental education enrollment was available 
only for the student’s first year enrolled after the student graduated from high school. 

Note: Fifteen percent of students participated in more than one program type. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 


Table C10. Relationship of accelerated college credit program participation to developmental education 
course enrollment in first year at a Rhode Island public college: Relative risk ratios, 2013/14 grade 9 cohort 


NV F-Var-le)(=) Relative risk Sy rlalel-lae) 
ratio error 

Base outcome: Enroll in a Rhode Island public college, no developmental education 

courses 

Outcome 1: Enroll in a Rhode Island public college, take developmental education 

course(s) 
Participation in any accelerated college credit program 0.35*** 0.04 
NECAP math score 0.90*** 0.01 
Eligible for National School Lunch Program 1.36* 0.17 
Racial/ethnic minority 0.92 0.14 
School: Percent students eligible for National School Lunch Program 1.39 0.61 
School: Percent multilingual learner students 0.42 0.58 
School: Percent students with IEP 1.03 1.15 
Intercept 0.81 0.21 

Outcome 2: Enroll in another type of college 
Participation in any accelerated college credit program 1.18 0.11 
NECAP math score 0.99 0.01 
Eligible for National School Lunch Program 0.73* 0.09 
Racial/ethnic minority 1.05 0.16 
School: Percent students eligible for National School Lunch Program 0.27** 0.11 
School: Percent multilingual learner students 0.78 0.93 
School: Percent students with IEP 2.09 2.21 
Intercept 1.16 0.25 

Outcome 3: Not enroll in college 
Participation in any accelerated college credit program 0.15*** 0.02 
NECAP math score 0.93*** 0.01 
Eligible for National School Lunch Program 1.65** 0.25 
Racial/ethnic minority 0.85 0.12 
School: Percent students eligible for National School Lunch Program 1.10 0.48 
School: Percent multilingual learner students 4.00 4.56 
School: Percent students with IEP 1.83 2.59 
Intercept 1.98* 0.53 


* p< .05; ** p< .01; *** p< .001. 

IEP is individualized education program. NECAP is New England Common Assessment Program. 

Note: The sample size is 3,916 students in a matched sample drawn from students who were first-time ninth graders in Rhode Island public schools in 
2013/14. Developmental education course enrollment was available only for students who enrolled in a public college in the state, and only for the student’s 
first year enrolled after the student graduated from high school. Relative risk ratios were obtained using a multinomial logistic regression model to estimate 
the probability of four postsecondary outcomes on the full sample of 3,916 students: (1) enrolling in a Rhode Island public college and taking developmental 
education courses, (2) enrolling in another type of college, and (3) not enrolling in college. The fourth outcome, enrolling in a Rhode Island public college 
and not taking developmental education courses, was the base outcome category against which the relative risk of each other outcome was compared. The 
relative risk ratio for accelerated college credit program participation for outcome 1 estimates the treatment effect on the relative likelihood of enrolling in 
developmental education courses in a Rhode Island public college versus enrolling in a Rhode Island public college but not taking developmental education 
courses, in the context of the likelihood of the two other outcomes. The regression model was a single-level model with cluster-adjusted standard errors to 
account for the clustering of students within high schools. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 
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Table C11. Relationship of accelerated college credit program participation to developmental education 
course enrollment in first year at a Rhode Island public college: Predicted probabilities, 2013/14 grade 9 
cohort 


Enroll in a Rhode Island Enroll in a RI public Enroll in another type of Not enroll in college 
ro}b) 0) (om exe) | (-¥-4-Fm ale) college, take (oro) | (4242) 
developmental education developmental 
courses (eYo[ Ulery ale) amaol¥] acx=) (<9) 
Probability Syelalerlae! Probability | Standard Probability Standard Probability Syelalerlae! 
error error error error 
Participation in any accelerated college credit program 
Yes 0.39*** 0.01 0.14*** 0.01 0.32*** 0.02 0.15*** 0.01 
No 0.21 0.01 0.17 0.01 0.16 0.01 0.46 0.02 


* p< .05; ** p< .01; *** p< .001. 

Note: The sample size is 3,916 students in a matched sample drawn from students who were first-time ninth graders in Rhode Island public schools in 
2013/14. Developmental education course enrollment was only available for students who enrolled in a public college in the state, and only for the student’s 
first year enrolled after graduating from high school. Predicted probabilities were obtained using a multinomial logistic regression model (see table C10) to 
estimate the probability of four postsecondary outcomes on the full sample of 3,916 students: (1) enrolling in a Rhode Island public college and taking 
developmental education courses, (2) enrolling in another type of college, and (3) not enrolling in college. A fourth outcome, enrolling in a Rhode Island 
public college and not taking developmental education courses, was the base outcome category against which the relative risk of each other outcome was 
compared. The predicted probability associated with accelerated college credit program participation status for outcome 1 shows the treatment effect on 
the relative likelihood of enrolling in developmental education courses in a Rhode Island public college versus enrolling in a Rhode Island public college but 
not taking developmental education courses, in the context of the likelihood of the two other outcomes. The regression model was a single-level model with 
cluster-adjusted standard errors to account for the clustering of students within high schools. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 


Table C12. Odds ratios for interaction effects for treatment and economic disadvantage, 2013/14 grade 9 
cohort 


Outcome 
High school graduation tal ae) faat=vaia la mere) (2124) 
NV FTar-le)(=) Odds ratio Standarderror Oddsratio Standard error 
Participation in any accelerated college credit program 7.12*** 1.32 4.58*** 0.49 
Eligible for National School Lunch Program 0.64** 0.09 0.56*** 0.06 
Participation by eligibility [interaction term] 1.50 0.41 1.35 0.22 


* p<.05; ** p<..01; *** p< 001. 

Note: The sample size is 3,916 students clustered in 57 high schools in a matched sample drawn from students who were first-time ninth graders in Rhode 
Island public schools in 2013/14. Shown are coefficients from two-level logistic regression models, fit separately for each of the two outcomes, that included 
student and school covariates (not shown in this table) as listed in table B1, appendix B. 

Source: Authors’ analysis of data from the Rhode Island Department of Education. 


Table C13. Predicted probabilities for interaction effects for treatment and economic disadvantage, 2013/14 


grade 9 cohort 
Outcome: 


alld ame alole) -4g-lelUF-yaKoy a) Enrollment in college 
Probability Syieloler-lacm-lagele Probability SVi-lerer-lacm-lanele 


Participated in any accelerated college credit 


program 
Eligible for National School Lunch Program 0.95*** 0.01 0.82*** 0.01 
Not eligible 0.96 0.01 0.86 0.01 
Did not participate 
Eligible for National School Lunch Program 0.71*** 0.03 0.45*** 0.02 
Not eligible 0.78 0.02 0.59 0.02 


* p< 105) ** pi<.01; *** p< 001. 

Note: The sample size is 3,916 students clustered in 57 high schools in a matched sample drawn from students who were first-time ninth graders in Rhode 
Island public schools in 2013/14. Predicted probabilities are estimated from the regression analysis displayed in table C12. The model included student and 
school covariates (not shown in this table) as listed in table B1, appendix B. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 


Table C14. Summary of odds ratios from models examining the relationship between different forms of 
accelerated college credit program participation and outcomes, 2013/14 grade 9 cohort 
Outcome: 


alt 4amsxeialole) M-4g-le [0 l-laLo)a) Enrollment in college 
Treatment indicator Odds ratio Standard error Odds ratio Standard error 
Ever enrolled in a dual enrollment course 4.39*** 1.38 2.73*** 0.50 
Ever enrolled in a concurrent enrollment course 4.68*** 0.81 3.60*** 0.36 
Ever took an Advanced Placement test 6.14*** 1.17 4.69*** 0.53 


* p< .05; ** p< .01; *** p< .001. 

Note: The sample size is 3,916 students clustered in 57 high schools in a matched sample drawn from students who were first-time ninth graders in Rhode 
Island public schools in 2013/14. Shown are estimated coefficients from two-level logistic regression models, fit separately for the two outcomes. For each 
outcome, the model included a set of three treatment indicators—one for each accelerated college credit program. Students could participate in more than 
one program. The models summarized above included student and school covariates as listed in table B1, appendix B. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 


Table C15. Summary of predicted probabilities from models examining the relationship between different 
forms of accelerated college credit program participation and outcomes, 2013/14 grade 9 cohort 


Outcome 

High school graduation Enrollment in college 
Treatment indicator Probability Standard error Probability Standard error 
Ever enrolled in a dual enrollment course 
Yes 0:95*** 0.01 0.83*** 0.02 
No 0.84 0.01 0.68 0.01 
Ever enrolled in a concurrent enrollment course 
Yes 0.95*** 0.01 0.84*** 0.01 
No 0.82 0.02 0.63 0.01 
Ever took an Advanced Placement test 
Yes 0.95*** 0.01 0.87*** 0.01 
No 0.82 0.02 0.63 0.01 


* p<.05; ** p<..01; *** p< 001. 

Note: The sample size is 3,916 students clustered in 57 high schools in a matched sample drawn from students who were first-time ninth graders in Rhode 
Island public schools in 2013/14. For each outcome, the model included a set of three treatment indicators, one for each accelerated college credit program. 
Students could participate in more than one program. Predicted probabilities are estimated from the two regression analyses displayed in table C14. The 
models summarized above included student and school covariates as listed in table B1, appendix B. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 


Table C16. Sensitivity analysis on sample excluding students departing the Rhode Island public school system 
before grade 11: Odds ratios from models examining the relationship between different forms of accelerated 
college credit program participation and outcomes, 2013/14 grade 9 cohort 


(OJU] Kore) aat=) 

High school graduation Enrollment in college 

Treatment indicator Odds ratio Standard error Odds ratio Standard 
error 

Model 1: 
Participated in any accelerated college credit 6.13*** 0.91 4.12*** 0.34 
program 
Model 2: 
Ever enrolled in a dual enrollment course 3.41*** 1.16 2.49*** 0.47 
Ever enrolled in a concurrent enrollment course 3.40*** 0.62 3.01*** 0.31 
Ever took an Advanced Placement test 4.92*** 1.03 3.798"? 0.43 


* p< (05; ** p<.01; *** p< 001. 

Note: The sample size is 3,696 students clustered in 58 high schools in a matched sample drawn from students who were first-time ninth graders in Rhode 
Island public schools in 2013/14 and had a high school enrollment record in 2015/16 and/or 2016/17. Shown are estimated coefficients from two-level 
logistic regression models, fit separately for the two outcomes. For participation in any program, a single binary indicator was entered in the model to 
represent treatment. For models analyzing outcomes associated with each program, the model included a set of three treatment indicators, one for each 
accelerated college credit program. Students could participate in more than one program. The models summarized above included student and school 
covariates as listed in table B1, appendix B. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 
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Table C17. Odds ratios for interaction effects for treatment and race/ethnicity, 2013/14 grade 9 cohort 
(OTE ] Kero) ant=) 


alt amsyelalole) -4g-le Ulva Koya) Enrollment in college Enrolled in developmental 
education in first year of 


(oo) | K=yeX-m aly a Valelelm if-1ale| 


Variable Odds Syelalerlae! (Ofolo Kw g-hd(o mms) f-] aler- a0] Odds ratio Syilaler-lae! 
ratio error error error 

Participation in any accelerated 9.20*** 1.79 4.55*** 0.46 1.65** 0.25 

college credit program 

Racial/ethnic minority 0.74** 0.11 1.03 0.13 0.85 0.12 

Participation by racial/ethnic 0.89 0.24 1.43* 0.24 2.09*** 0.42 


minority [interaction term] 


* p< 105; ** p<..01: *** p< 001. 

Note: The sample size is 3,916 students clustered in 57 high schools in a matched sample drawn from students who were first-time ninth graders in Rhode 
Island public schools in 2013/14. Shown are coefficients from two-level logistic regression models, fit separately for each of the two outcomes, that included 
student and school covariates (not shown in this table) as listed in table B1, appendix B. 

Source: Authors’ analysis of data from the Rhode Island Department of Education. 


Table C18. Predicted probabilities for interaction effects for treatment and minority status, 2013/14 grade 9 


cohort 
Outcome 


High school graduation Enrollment in college Enrolled in 
developmental education 


in first year of college in 
i Varere l= ks F-lale| 
Probability Syelalerlae! Probability Slater] ae] Probability Sii-laler-lae| 
error error error 


Participated in any 
accelerated college credit 


program 
Racial/ethnic minority 0.96*** 0.01 0.82*** 0.01 0.15*** 0.01 
White 0.95 0.01 0.87 0.01 0.16 0.01 
Did not participate 

Racial/ethnic minority 0.72*** 0.03 0.54*** 0.02 0.12*** 0.01 
White 0.77 0.02 0.53 0.02 0.18 0.01 


* p< .05; ** p< .01; *** p< .001. 

Note: The sample size is 3,916 students clustered in 57 high schools in a matched sample drawn from students who were first-time ninth graders in Rhode 
Island public schools in 2013/14. Predicted probabilities are estimated from the regression analysis displayed in table C12. The model included student and 
school covariates (not shown in this table) as listed in table B1, appendix B. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 


Appendix D. Other analyses 


The Rhode Island Department of Education (RIDE) requested information about two additional cohorts of 
students: the grade 9 cohorts of 2014/15 and of 2015/16. These cohorts were not included in the analyses 
featured in this report because data were incomplete. The 2015/16 grade 9 cohort did not have a baseline 
standardized assessment score available for grade 8, because the state transitioned from the New England 
Common Assessment Program (NECAP) to the Partnership for Assessment of Readiness for College and Careers 
assessments that year. In addition, both classes had graduated too recently for college enrollment data to be 
available to the state longitudinal data system. 


Because detailed student-level information about dual and concurrent enrollment was not reported to the state 
agency until 2015/16, these two cohorts provide a more complete picture of enrollment in these programs than 
did the 2013/14 grade 9 cohort featured in the report. The 2014/15 grade 9 cohort has dual and concurrent 
enrollment data for grades 10—12, and the 2015/16 grade 9 cohort has data for the full high school grade span of 
grades 9-12 (table D1). 


Table D1. Accelerated college credit program student characteristics, 2014/15 and 2015/16 grade 9 cohorts 
All (Go) | [-1-X-morg=te limo) goy-4e- lean ey) ad iol] oY-14le) 4) 


students Any college credit Any concurrent Bele) a-Tanar Ne Velalectel 
Grade 9 cohort — j,, cohort 
program Any dual enrollment enrollment Placement test 
N N Percent N Percent N Percent N Percent 
2014/15 9,172 4,132 45.1 450 49 2,476 27.0 2,914 31.8 
2015/16 9,997 4,517 45.2 517 5.2 2,774 27.9 3,199 32.0 


AP is Advanced Placement. N is number of students. 
Source: Authors’ analysis of data from Rhode Island Department of Education. 
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Tables D2 through D5 display the characteristics of students in the 2014/15 and 2015/16 grade 9 cohorts. 


Table D2. Accelerated college credit program student characteristics, 2014/15 grade 9 cohort 


(Go) | -¥-x-Moig=tol im )ge)-4g-1 nam oy-]ad iol] oy-a (ola) 


Took any 

Characteristics in All students in Na Were) |(-14-mol gle ht Any dual Any concurrent P NeW 7-Tarercxe| 
grade 8 program enrollment enrollment Placement test 

N Percent Percent Percent 
Male 4,719 51.5 1,800 43.6 164 36.4 1,102 44.5 1,266 43.5 
Rebielcthate 3350 365 1,300 315 236 52.4 641 25.9 882 30.3 
minority 
National School 
Lunch Program 4,548 49.6 1,473 35.7 267 59.3 767 31.0 949 32.6 
eligible 
Has IEP 1,293 14.1 156 3.8 31 6.9 91 3.7 60 2.1 
MalHlinetal 480 59 125 3.0 19 4.2 68 2.8 70 2.4 
learner 
nee oe mere 897 9.8 118 2.9 33 a3 56 2.3 59 2.0 
suspensions 
oe eel 1,482 16.2 297 7.2 68 15.1 160 6.5 160 5.5 
NECAP math: 
substantially 2,022 22.1 282 6.8 71 15.8 148 6.0 125 4.3 
below Proficient 
Partially Proficient 1,858 20.3 513 12.4 82 18.3 296 12.0 250 8.6 
Proficient 3,730 40.7 2,056 49.8 215 47.9 1,252 50.6 1,397 48.0 
Ponies with 1559 17.0 41,280 31.0 81 18.0 780 315 1,141 39.2 
Distinction 


AP is Advanced Placement. IEP is individualized education program. N is number of students. NECAP is New England Common Assessment Program. 

Note: The sample is 9,172 students who were first-time ninth graders in 2014/15. For NECAP, the sample is 9,169 because there are three cases with NECAP 
scores but no proficiency level. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 
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Table D3. Accelerated college credit program students by characteristics of their first high school, 2014/15 
grade 9 cohort 


(Go) | (-¥-x-Moig-tolimolge)-4e-)eam ox-]ad iol] oy-a(ol a) 


Took any 

Yel afere)| All students in Any college credit Any dual Any concurrent rNoN7-Taler-xo| 
characteristics (oto) areola program enrollment enrollment Placement test 

Percent N Percent Percent Percent 
Locale 
Rural 783 8.6 402 9.7 33 7.3 263 10.6 273 9.4 
Suburban 6,149 67.0 2,819 68.2 190 42.2 1,818 73.4 2,025 69.5 
Urban 2,050 22.3 880 21.3 225 50.0 372 15.0 594 10.4 
Urban ring 190 2.1 31 0.8 2 0.4 23 0.9 22 0.8 
Driving distance to nearest Rhode Island public college campus 
<=1mile 
ie 657 7.2 247 6.0 121 26.9 72 2.9 108 3.7 
distance) 
1-8 miles 5,848 63.8 2,546 61.6 259 57.6 1,580 63.8 1,821 62.5 
>8 miles 2,667 29.1 1,339 32.4 70 15.6 824 33.3 985 33.8 
Enrollment 
Small 821 9.0 367 8.9 68 15.1 198 8.0 221 7.6 
Medium 6,081 66.3 2,823 68.3 309 68.7 1,593 64.3. 2,109 72.4 
Large 2,270 24.8 942 22.8 73 16.2 685 27.7 584 20.0 
Star school accountability rating (5 indicates best rating) 
1 1,400 15.7 436 10.7 61 13.8 172 7.0 322 11.2 
2 1,909 21.5 728 17.9 170 38.6 457 18.6 394 13.7 
3 2,829 31.8 1,168 28.7 91 20.6 737 30.1 846 29.5 
4 2,239 25.2 1,288 31.6 101 22.9 870 35.5 916 31.9 
5 516 5.8 456 11.2 18 4.1 216 8.8 392 13.7 


AP is Advanced Placement. N is number of students. 

Note: The sample is 9,172 students who were first-time ninth graders in 2014/15. For Star rating, the sample is 8,893 because 279 students attended schools 
with no Star rating. State agencies in Rhode Island commonly use the term “urban” to refer to the cities of Central Falls, Newport, Pawtucket, Providence, 
and Woonsocket, and “urban ring” to refer to the surrounding municipalities of Cranston, East Providence, North Providence, Warwick, and West Warwick. 
Enrollment categories are defined thus: small equals less than 500 students, medium equals 500-1,199, and large equals 1,200 or higher. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 
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Table D4. Accelerated college credit program student characteristics, 2015/16 grade 9 cohort 


(Go) | (-¥-x-merg=tol im ol coy-4e-] am oy-Vad lol] oy-a(o) a) 


Characteristics in All students in Any college credit Any dual Na Wactolaleelansvals Took any Advanced 
grade 8 cohort program enrollment enrollment Placement test 
Percent Percent Percent Percent Percent 
Male 5,129 51.3 1,901 42.1 140 27.1 1,195 43.1 1,347 42.1 
Racial/ethnic 
; / 3,868 38.7. 1,556 34.5 291 56.3 791 28.5 1,083 33.9 
minority 


National School 


Lunch Program 4860 48.6 1,582 35.0 308 59.6 830 29.9 1,013 31.7 
eligible 

Has IEP 1,338 13.4 170 3.8 32 6.2 97 3.5 72 2.3 
Mulnilingdel 618 6.2 191 4.2 36 7.0 88 3.2 130 4.1 
learner 

Has 1 or more 942 9.4 150 3.3 38 7.4 72 2.6 74 2.3 
suspensions 

CHFONIGADSENCE- 4 494 18.2 382 8.5 87 16.8 200 7.2 210 6.6 


(10+) 


AP is Advanced Placement. IEP is individualized education program. N is number of students. NECAP is New England Common Assessment Program. 
Note: The sample size is 9,997 students who were first-time ninth graders in 2015/16. Grade 8 NECAP scores are not available for students in this cohort. 
Source: Authors’ analysis of data from Rhode Island Department of Education. 


Table D5. Accelerated college credit program school characteristics, 2015/16 grade 9 cohort 


(@(o) | (-¥-X-Molg=toli mo) ge)-4e-]e0M ey-)aa(elley-1alela 


Kel sfere)| All students in Na Waere)|(-14-mel gle hi Any dual Any concurrent 

characteristics (oto) aYelat program enrollment enrollment Took any AP test 
Percent Percent Percent Percent 

Locale 

Rural 819 8.2 404 8.9 18 3.5 236 8.5 315 9.9 

Suburban 6,603 66.1 3,029 67.1 221 42.8 2,048 73.8 2,126 66.5 

Urban 2,393 12.9 1,041 23.1 277 53.6 459 16.6 724 22.6 

Urban ring 182 1.8 43 1.0 1 0.2 31 1.1 34 1.4 

Driving distance to nearest Rhode Island public college campus 

<=1mile 

wae 772 7.7 295 6.5 137 26.5 74 2.7 162 5.1 

distance) 

1-8 miles 6,457 64.6 2,929 64.8 322 62.3 1,915 69.0 2,052 64.2 

> 8 miles 2,768 27.7 1,293 18.6 58 11.2 785 28.3 985 30.8 

Enrollment 

Small 1,186 11.9 500 11.1 101 19.5 291 10.5 299 9.4 

Medium 6,326 63.3 2,949 65.3 300 58.0 1,690 60.9 2,220 69.4 

Large 2,485 24.9 1,068 23.6 116 22.4 793 28.6 680 21.3 

Star school accountability rating (5 indicates best rating) 

a 1,657 17 A 574 13.0 80 15.9 270 9.9 471 13.3 

2 2,059 21.2 751 16.9 174 34.5 457 16.7 441 14.1 

3 3,110 32.0 1,281 28.9 109 21.6 810 29.6 912 29.1 

4 2,409 24.8 1,410 31.8 112 22.2 987 36.0 1,012 32.3 

5 480 4.9 418 9.4 29 5.8 216 7.9 349 11.2 


AP is Advanced Placement. N is number of students. 

Note: The sample is 9,997 students who were first-time ninth graders in 2014/15. For Star rating, the sample is 9,715 because 282 students attended schools 
with no Star rating. N is number of students. State agencies in Rhode Island commonly use the term “urban” to refer to the cities of Central Falls, Newport, 
Pawtucket, Providence, and Woonsocket, and “urban ring” to refer to the surrounding municipalities of Cranston, East Providence, North Providence, 
Warwick, and West Warwick. Enrollment categories are defined thus: small equals less than 500 students, medium equals 500—1,199, and large equals 1,200 
or higher. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 
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Table D6 displays descriptive statistics for the attainment of the one outcome—high school graduation—available 
for both cohorts. 


Table D6. Percentage of students graduating from high school, 2014/15 and 2015/16 grade 9 cohorts 


2014/15 cohort 2015/16 cohort 
(N = 9,172) (N = 9,997) 

Graduated from high school: Number Percent Number Percent 
All students in cohort 7,646 83.4 8,151 81.5 
Participation in any accelerated college credit program 4,069 98.5 4,409 97.6 
No participation in any accelerated college credit program 3,577 71.0 3,742 68.3 
Specific programs: 
Ever enrolled in a dual enrollment course 437 97.1 500 96.7 
Ever enrolled in a concurrent enrollment course 2,446 98.8 2,715 97.9 
Ever took an Advanced Placement test 2,891 99.2 3,151 98.5 


N is number of students. 
Source: Authors’ analysis of data from Rhode Island Department of Education. 


The research team applied the same propensity score estimation and matching specifications to the 2014/15 
grade 9 cohort that were used for the 2013/14 cohort (see appendix B). Baseline equivalence after matching was 
not as satisfactory for this cohort, with the following covariates having standardized mean differences greater 
than 0.05 and less than 0.25: grade 8 NECAP mathematics score (-0.06), percentage of students in the school 
eligible for the National School Lunch Program (0.06), percentage of students in the school who were multilingual 
learners, and percentage of students in the school with an individualized education program (0.08). As noted in 
appendix B, these four measures were included in all treatment estimation regression models for all cohorts to 
minimize selection bias. 


The research team attempted to create a matched sample in the 2015/16 grade 9 cohort but could not produce 
an acceptable level of baseline equivalence. The lack of a baseline standardized academic achievement measure 
may have contributed to the difficulty in achieving balance on other correlated baseline measures. Because 
matching did not produce appreciable benefits in the reduction of selection bias, the research team performed 
regression analyses on the unmatched sample of all cohort members, using the same model specifications as 
described for the 2013/14 grade 9 cohort in appendix B. 


The results of regression analyses on these two additional cohorts are shown in tables D7 and D8. 
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Table D7. Summary of odds ratios from models examining the relationship between different forms of 
accelerated college credit program participation and high school graduation, 2014/15 and 2015/16 grade 9 
cohorts 


PONY Abo w-{e-\ol-me Mere) alolae 2015/16 grade 9 cohort (not a 
[rate] meiat=Yocr-]a0)¢)(=)) matched sample) 
Treatment indicator ofole Km g-hale) Sielalerlae| Odds ratio Sielalerlae! 
error error 
Participated in any accelerated college credit 15.41*** 2.56 18.15*** 1.92 
program 
Model with specific program indicators: 
Ever enrolled in a dual enrollment course 4.28*** 1.41 6.46*** 1.70 
Ever enrolled in a concurrent enrollment course 9.96*** 2.41 7.61*** 1.07 
Ever took an Advanced Placement test 19.10*** 5.72 15.24*** 2.35 


Note: For the 2014/15 9th-grade cohort, the sample size is 4,152 students in 57 high schools in a matched sample drawn from students who were first-time 
ninth graders in Rhode Island public schools in 2014/15. For the 2015/16 ninth grade cohort, the sample size is 9,997 students in 65 high schools drawn from 
students who were first-time ninth graders in Rhode Island public schools in 2015/16. The latter cohort sample is not matched because a baseline grade 8 
standardized assessment score was not available. Odds ratios were estimated from multilevel logistic regression models. The models summarized in this 
table included student and school covariates as listed in table B1, appendix B. 

Source: Authors’ analysis of data from Rhode Island Department of Education. 


Table D8. Summary of predicted probabilities from models examining the relationship between different 
forms of accelerated college credit program participation and high school graduation, 2014/15 and 2015/16 
grade 9 cohorts 


2014-2015 grade 9 2015-2016 grade 9 
cohort (matched sample) (oto) aXe) am (aXe) ar-Maateinealsyo| 
sample) 
Treatment indicator Probability Standard Probability | Standard 
error error 

Participated in any accelerated college credit program 
Yes 0.98*** 0.00 0.97*** 0.00 
No 0.76*** 0.01 0.69*** 0.01 
Ever enrolled in a dual enrollment course 
Yes 0.96*** 0.01 0.96*** 0.01 
No 0.87*** 0.01 0.80*** 0.01 
Ever enrolled in a concurrent enrollment course 
Yes 0.98*** 0.01 0.96*** 0.01 
No 0.84*** 0.01 0.78*** 0.01 
Ever took an Advanced Placement test 
Yes 0.98*** 0.00 0.98*** 0.00 
No 0.83*** 0.01 0.77*** 0.01 


Note: For the 2014/15 grade 9 cohort, the sample size is 4,152 students in 57 high schools in a matched sample drawn from students who were first-time 
ninth graders in Rhode Island public schools in 2014/15. For the 2015/16 grade 9 cohort, the sample size is 9,997 students in 65 high schools drawn from 
students who were first-time ninth graders in Rhode Island public schools in 2015/16. The latter cohort sample is not matched because a baseline grade 8 
standardized assessment score was not available. For each outcome, the model included a set of three treatment indicators, one for each accelerated college 
credit program. Students could participate in more than one program. Predicted probabilities were estimated using multilevel logistic regression models 
(see table D7). The models summarized above included student and school covariates as listed in table B1, appendix B. 

Source: Authors’ analysis of data from the Rhode Island Department of Education. 


Table D9 displays more detailed information about the high schools in which students participated in dual and 
concurrent enrollment programs. The 2015/16 grade 9 cohort is used because this cohort had complete 
participation data for grades 9-12. 


Table D9. Schools in which at least 10 students participated in dual enrollment in the 2015/16 grade 9 cohort 


All schools represented in lta) ce) | (Xe i= jm (=r-k] a A) Enrolled at least 10 
idatemeo) alo) aumcr-lan}e)( students from the cohort in students from the cohort in 
(N = 65) dual enrollment concurrent enrollment 
(N = 16) (N = 46) 

Locale 
Urban 22 33.85 8 50.00 12 26.09 
Suburban 34 52.31 8 50.00 27 58.70 
Rural 8 12.31 0 0.00 6 13.04 
Urban-ring 1 1.54 0 0.00 1 2.17 
Type 
Comprehensive 46 70.77 8 50.00 39 84.78 
Vocational-technical 8 12.31 4 25.00 3 6.52 
Charter 11 16.92 4 25.00 4 8.70 
Total enrollment 
Large 7 10.77 4 25.00 7 15.22 
Medium 35 53.85 8 50.00 32 69.57 
Small 23 35.38 4 25.00 7 15.22 


N is number of students. 
Note: Enrollment categories are defined thus: small equals less than 500 students, medium equals 500-1,199, and large equals 1,200 or higher. 
Source: Authors’ analysis of data from Rhode Island Department of Education. 


